Safe Reinforcement Learning via Shielding under Partial Observability

Safe exploration is a common problem in reinforcement learning (RL) that aims to prevent agents from making disastrous decisions while exploring their environment. A family of approaches this assume domain knowledge the form (partial) model environment decide upon safety an action. so-called shield forces RL agent select only safe actions. However, for adoption various applications, one must look beyond enforcing and also ensure applicability with good performance. We extend shields via tight integration state-of-the-art deep RL, provide extensive, empirical study challenging, sparse-reward environments under partial observability. show carefully integrated ensures can improve convergence rate final performance agents. furthermore be used bootstrap agents: they remain after initial shielded setting, allowing us disable potentially too conservative eventually.